An image sensing apparatus such as a small video camera has a disadvantage that, because the apparatus shakes due to so-called camera shake or vibration when sensing an image, a fuzzy image is output or recorded. Therefore, as measures for eliminating such a disadvantage, an image sensing apparatus provided with a vibration correcting function which reduces influences of camera shake has been developed and already commercialized.
There are various methods for detecting vibrations such as camera shake, for example, a method of directly detecting motion of an apparatus using an angular velocity sensor or angular acceleration sensor, and an electronic detection method for detecting motion of an image by comparing between images of successive fields or frames among image signals. On the other hand, there are means for correcting vibration, for example, one provided with a vibration correction optical system which optically adjusts an angle of the image sensing optical axis in a direction in which camera shake is cancelled and so-called electronic correcting means which electronically selects a range to be actually recorded or output (extraction range) of a sensed image by an image sensing element.
A conventional example using an angular velocity sensor as vibration detecting means and using a vibration correction optical system as vibration correcting means will be explained below.
FIG. 9 is a conceptual diagram of an image sensing optical system including a vibration correction optical system.
In FIG. 9, an image sensing optical system 700 includes a fixed lens 701 which is fixed to a lens barrel (not shown), a zoom lens 702 which moves on a central axis c′ of the image sensing optical system 700 in horizontal direction as indicated by an arrow a, a shift lens 703 which moves two-dimensionally within the plane (direction indicated by an arrow b) which is perpendicular to the central axis c′, a focus lens 704 which corrects the movement of a focal plane due to a focusing function and movement of the zoom lens 702, and an image sensing element 705 which forms an image of an object, arranged in the foregoing order, and is further provided with an actuator 706 which drives the shift lens 703 and a position detection sensor 707 which detects the position of the shift lens 703 at predetermined positions near the shift lens 703.
As shown in FIG. 10A, when camera shake, etc., causes the optical axis c to deviate from the central axis c′ of the image sensing optical system 700, producing a displacement angle θ, it is possible, by driving the actuator 706 and moving the shift lens 703 to the position indicated by 703′ as shown in FIG. 10B, to optically align the optical axis c which is deviated on the fixed lens 701 side with respect to the shift lens 703 with the central axis c′ of the image sensing optical system on the image sensing element 705 side with respect to the shift lens 703. Therefore, it is possible to correct the optical displacement angle θ produced by camera shake as described above through the above described operation and form an image of the object on the image sensing element 705 as an incident beam with vibration corrected by moving the shift lens 703 based on the camera shake.
Then, an example of a configuration of a conventional image sensing apparatus shown in pp4–6, and FIGS. 2 and 3 of Japanese Patent Laid-Open No. 2000-39637, and pp3–4 and FIG. 1 of Japanese Patent Laid-Open No. 2000-66259 is shown in FIG. 11.
In FIG. 11, the conventional image sensing apparatus is constructed of the aforementioned image sensing optical system 700 including the vibration correction system, an image sensing element 705 on which an optical image of an object is formed through the image sensing optical system 700, a camera signal processing circuit 1519 which applies predetermined signal processing to the output from the image sensing element 705, an angular velocity sensor 1501 which detects vibration of the apparatus, a high pass filter (hereinafter simply referred to as “HPF”) 1502 which removes a direct current (DC) component from the output of the angular velocity sensor 1501, a first amplifier 1503 which amplifies the output from the HPF 1502 by a predetermined amount, a microcomputer 1505 which applies predetermined signal processing to the output from the first amplifier 1503, a D/A converter 1515 which converts the output from the microcomputer 1505 to an analog signal, a driving circuit 1517 which issues a driving signal of the actuator 706 included in the image sensing optical system 700, a second amplifier 1518 which amplifies the output from the position detection sensor 707 by a predetermined amount and an adder 1516 which adds up the output from the D/A converter 1515 and the output from the second amplifier 1518.
In this configuration, the angular velocity sensor 1501 outputs a vibration detection signal based on vibration of the apparatus, the vibration detection signal HPF 1502 removes the DC component from it, and then the first amplifier 1503 amplifies it by a predetermined amount. That is, the configuration from the angular velocity sensor 1501 to the first amplifier 1503 causes the vibration detection signal from the angular velocity sensor 1501 to convert to a vibration detection signal processed with predetermined band restriction and amplification and the vibration detection signal is input to the microcomputer 1505 which controls the image sensing apparatus. The vibration detection signal input to the microcomputer 1505 is subjected to predetermined signal processing to calculate a control amount of vibration correction (hereinafter simply referred to as a “correction target value”). This predetermined signal processing will be described later.
Then, the correction target value calculated by the microcomputer 1505 is converted from a digital signal to an analog signal at the D/A converter 1515, input to the adder 1516 and added to a feedback signal from the position detection sensor 707 of the shift lens 703 supplied through the second amplifier 1518. The output signal from the adder 1516 is supplied to the driving circuit 1517 and the driving circuit 1517 issues a driving signal to the actuator 706 and drives the shift lens 703. This allows the displacement θ to be optically corrected as explained in FIGS. 10A and 10B, causing the object image to be formed on the image sensing element 705 as a beam with vibration corrected.
Then, the electric signal photoelectrically converted by the image sensing element 705 is led through a camera signal processing circuit 1519 and supplied to a recording/reproducing section (not shown), etc.
Next, the signal processing in the microcomputer 1505 will be explained.
FIG. 12 shows a signal processing system in the microcomputer 1505, including an A/D converter 1506 which converts the input vibration detection signal from an analog signal to a digital signal, an HPF 1507 which removes a DC component from the output of the A/D converter 1506, a phase compensation section 1508 which phase-compensates the output of the HPF 1507, a variable HPF 1509 which restricts the pass band of the output of the phase compensation section 1508, a first integrator 1510 which integrates the output of the variable HPF 1509, a frequency detection section 1511 which detects the vibration frequency from the output of the A/D converter 1506 through the HPF 1507 and a vibration correction frequency control section 1514 which decides the vibration state of the apparatus from the output of the frequency detection section 1511 and controls the frequency for correcting the vibration. The frequency detection section 1511 includes a second integrator 1512 which integrates the output of the A/D converter 1506 which has passed through the HPF 1507 and a frequency calculation section 1513 which calculates the frequency from this integrated output.
In the above described configuration, the input vibration detection signal is converted at the A/D converter 1506 from an analog vibration signal to a digital vibration signal and then remove the DC component generated through A/D conversion, etc., at the HPF 1507. Therefore, the cutoff frequency of the HPF 1507 is sufficiently low. Then, at the phase compensation section 1508, the vibration detection signal from which the DC component is removed is phase-compensated for a phase delay in a high frequency band in such a way that the phase characteristic becomes flat up to a predetermined frequency band, then subjected to predetermined pass band restriction and phase compensation which will be described later at the variable HPF 1509 whose cutoff frequency is variable, further subjected to integration processing at the first integrator 1510 to convert the angular velocity signal to an angular displacement signal whereby a correction target value is obtained and supplied to the D/A converter 1515.
Furthermore, the output of the HPF 1507 is input to the phase compensation section 1508 as shown in FIG. 12 and at the same time also input to the frequency detection section 1511, where the vibration frequency of the apparatus is detected. The detection of the vibration frequency will be described later.
Then, the detected vibration frequency is input to the vibration correction frequency control section 1514, where a cutoff frequency is selected from table data corresponding to the vibration frequency from the frequency detection section 1511 and set in the variable HPF 1509. More specifically, control is performed in such a way that the cutoff frequency remains at a specified value or the cutoff frequency is shifted gradually from the cutoff frequency of a specified value to the high frequency side or the cutoff frequency is returned gradually from a state in which it has been shifted to the high frequency side to the cutoff frequency of a specified value (hereinafter simply referred to as “adaptive control”) and the signal is phase-compensated for a phase delay in the high frequency band which cannot be phase-compensated by the phase compensation section 1508.
Then, the detection of a vibration frequency will be explained.
As shown in FIG. 12, the frequency detection section 1511 includes the second integrator 1512 and frequency calculation section 1513. The second integrator 1512 integrates the output of the A/D converter 1506 which has passed through the HPF 1507, thereby converts the angular velocity signal to an angular displacement signal and calculates a second angular displacement signal. Based on the above described calculated second angular displacement signal, the frequency calculation section 1513 calculates the frequency and detects the vibration frequency of the apparatus.
Next, the calculation of an angular displacement signal for frequency detection and calculation of the frequency will be explained.
FIG. 13 shows an input/output characteristic of the second integrator 1512 which calculates an angular displacement signal to calculate the vibration frequency of the apparatus, which shows the frequency on the abscissa and gain on the ordinate.
As is apparent from FIG. 13, the output of the second integrator 1512 has an integration characteristic in which the output is greater in a low frequency band and smaller in a high frequency band. Therefore, the high frequency band which is mixed with the output of the HPF 1507 input to the second integrator 1512 attenuates and the angular displacement signal of the low frequency band at a large amplitude level is calculated.
Next, the operation of the frequency calculation section 1513 which calculates a vibration frequency of the apparatus from the calculated angular displacement signal will be explained using FIG. 14.
FIG. 14 is a flow chart showing frequency detection processing carried out in the microcomputer 1505 and rough description of this processing will be given first.
In step S1101 in this figure, frequency detection is started and in step S1102, the number of increase/decrease turning points of the vibration signal calculated by the second integrator 1512 is counted first. Then, in next step S1103 the count value is stored in a register and in step S1104 the count value is compared with a predetermined first threshold (th1). If the count value is equal to or lower than the predetermined first threshold (th1), the process moves on to step S1105, where it calculates a first frequency, then moves on to step S1108 and finishes the frequency detection.
On the other hand, if the count value is greater than the first threshold (th1) in step S1104, the process moves on to step S1106, where it compares the number of times (count value>th1) occurs consecutively with a predetermined second threshold (th2). As a result, if the number of times (count value>th1) is equal to or lower than the second threshold (th2), the process moves on to step S1108, where it finishes the frequency detection. If the number of times (count value>th1) is greater than the second threshold (th2), the process moves on to step S1107, where it calculates a second frequency and then moves on to step S1108 and finishes the frequency detection.
Then, the specific operation of the frequency detection will be explained using FIG. 14.
As the method for frequency detection, the number of increase/decrease turning points of the vibration signal per a unit time is counted and the counted number is regarded as the detected frequency.
In step S1101, frequency detection which is carried out at a period (e.g., 500 [ms]) longer than a vibration correction control processing period (e.g., 1 [ms]) is started. First in step S1102, the number of increase/decrease turning points of an angular displacement signal is counted whereby the increase/decrease subjected to the counting is a difference between previous sampling data and latest sampling data of an angular displacement signal sampled at a predetermined period (e.g., 10 [ms]) which exceeds a predetermined threshold. Then in next step S1103, the counted value is stored in a register. This register is a shift register constructed in such a way as to be able to store a plurality of sample data (n=x), shift data every time the count value is updated and erase the oldest data.
Then in step S1104, the latest count value (number of increase/decrease turning points) is compared with the first threshold (th1). For example, when the first threshold (th1) is set to “12” and the latest count value is “10”, the updated count value as a result of comparison becomes th1 or less (NO) and the process moves on to step S1105. Then, in this step S1105, the frequency per a unit time is calculated from the number of increase/decrease turning points “10” which is the latest count value stored in the register. The number of increase/decrease turning points in one period is 2 and 1 [Hz], that is, since the frequency is ½ of the number of increase/decrease turning points, that is “10/2=5” and a frequency of 5 [Hz] is calculated. After the frequency is calculated, the process moves on to step S1108 and finishes the frequency detection processing.
Furthermore, in above step S1104, if the latest count value (number of increase/decrease turning points) is higher than th1, for example if the first threshold (th1) is “12” and latest count value is “16”, the updated count value is higher than th1 (YES). In this case, the process moves on to step S1106, where the number of times that the comparison condition (count value>th1) in step S1104 holds consecutively is compared with the second threshold (th2). This processing is carried out to improve the reliability of counting because when the count in step S1102 increases sporadically due to noise, etc., the comparison condition (count value>th1) in step S1104 is satisfied.
When the number of times the comparison condition (count value>th1) in step S1104 is satisfied is equal to or lower than a predetermined threshold (th2) (No in step S1106), the process moves on to step S1108, where it finishes the processing of frequency detection. That is, the detected frequency is not updated.
On the other hand, when the number of times the comparison condition (count value>th1) in step S1104 is satisfied is greater than a predetermined threshold (th2) ((count value)>th1))>th2 holds (YES in step S1106), the process moves on to step S1107, where the latest count value stored in the register is compared with count values stored in the past and adopts a minimum value as the detected frequency. More specifically, assuming that the count values stored in the register are for example, 16, 18 and 18, that is, n=3, the microcomputer compares them and selects 16 as a minimum value. In this case, the frequency is 16/2=8 as described above and this means that a frequency 8 [Hz] is calculated.
The minimum value is regarded as the detection frequency because the camera shake frequency relatively tends to concentrate on a low frequency (several [Hz] to 10 [Hz]) and the cutoff frequency of the variable HPF 1509 is controlled based on the frequency detected assuming the use on a vehicle, etc., and therefore this is intended to reduce sacrificing of the vibration correction effect on the low frequency side to a lowest possible level even when the vibration correction frequency is shifted to the high frequency side. Then, the microcomputer moves on to step S1108 and finishes the processing of frequency detection.
Next, the operation of the vibration correction frequency control section 1514 which determines the vibration state of the apparatus according to the detected vibration frequency and sets the cutoff frequency of the variable HPF 1509 will be explained below.
A predetermined frequency threshold (fth) is set in the vibration correction frequency control section 1514 for the vibration frequency detected by the frequency detection section 1511. Therefore, the vibration correction frequency control section 1514 compares the detected vibration frequency with the predetermined frequency threshold (fth), decides whether the detected vibration frequency is higher than fth or not, and performs adaptive control, based on the determination result, such as to decide whether to continue to use the predetermined specified value as the cutoff frequency of the variable HPF 1509 or shift it from the specified value to the high frequency side gradually or return it from the state in which it has been shifted to the high frequency side to the specified value gradually.
Next, the frequency characteristic of the variable HPF 1509 when adaptive control is performed will be explained using FIG. 15A and FIG. 15B. FIG. 15A shows a gain characteristic and FIG. 15B shows a phase characteristic.
The variable HPF 1509 has a frequency characteristic up to the normal camera shake frequency band (e.g., approximately 3 to 8 [Hz]) indicated by a gain 1201 and phase 1202 set for a predetermined cutoff frequency fc and the cutoff frequency remains at the specified value fc. However, when the apparatus is fixed to a vehicle, etc., and the vehicle moves and when a frequency (e.g., approximately 20 [Hz]) exceeding the frequency of camera shake is detected, the vibration correction frequency control section 1514 controls so that the cutoff frequency of the variable HPF 1509 is shifted to the high frequency side gradually based on the detection frequency. The frequency characteristic when the cutoff frequency of the variable HPF 1509 is shifted gradually to the high frequency side is gain 1201′ and phase 1202′ in the case of the cutoff frequency fc′ shown in FIG. 15A and FIG. 15B. Furthermore, when the detected frequency changes from the frequency exceeding the frequency of camera shake to a normal frequency of camera shake, the vibration correction frequency control section 1514 controls so that the cutoff frequency fc′ of the variable HPF 1509 is gradually shifted to fc.
Thus, it is possible to make the cutoff frequency of the variable HPF 1509 variable through adaptive control. Since the phase characteristic when the cutoff frequency is shifted to the high frequency side (fc′) is a leading phase (1202′), phase compensation for the high frequency band which will be described later is performed.
Then, phase compensation for a phase delay of a high frequency band will be explained.
FIGS. 16A and 16B show a frequency characteristic from the angular velocity sensor 1501 to the output of the vibration correction system and reference numeral 1301 in FIG. 16A shows a gain characteristic and reference numeral 1302 in FIG. 16B shows a phase characteristic.
In FIGS. 16A and 16B, a frequency band 1303 shows a vibration correctable band and it is for example, frequency f1=1 Hz, f2=20 Hz and f3=30 Hz. The range of a band 1304 where the gain attenuates shows a band in which vibration correction is disabled. In the bands between frequencies f2 and f3, the phase shows a lag in the high frequency band in the vibration correctable range as shown in FIG. 16B. Therefore, the phase characteristic of the variable HPF 1509 which changes as the cutoff frequency of the variable HPF 1509 is shifted to the high frequency side (leading phase) makes it possible to phase-compensate the high frequency band in which a phase delay occurs and improves the vibration suppression effect of a high frequency band equal to or higher than the normal camera shake frequency by bringing the phase characteristic closer to flat.
The vibration signal that passes through the variable HPF 1509 which is adaptively controlled in this way is integrated by the first integrator 1510 whereby an angular displacement signal is output as a correction target value.
Next, the processing whereby the aforementioned content is executed in the microcomputer 1505 will be explained with reference FIGS. 17 and 18.
FIG. 17 is a flow chart of the vibration correction processing executed in the microcomputer 1505 and is the processing for interrupting the overall processing of the microcomputer 1505 in a predetermined period (e.g., 1 [ms]).
In FIG. 17, the process start by an interruption in step S1401 and an analog vibration detection signal captured by the A/D converter 1506 is converted to a digital vibration detection signal in step S1402 first. Then, in next step S1403, the HPF 1507 removes the DC component generated through A/D conversion. Then in next step S1404, the phase compensation section 1508 phase-compensates for a predetermined band of the vibration detection signal whose DC component has been removed. Then, in step S1405, the variable HPF 1509 applies predetermined band restriction on the vibration detection signal undergone predetermined phase compensation.
In next step S1406, the first integrator 1510 integrates the vibration detection signal undergone the predetermined band restriction to calculate a first angular displacement signal. Then, in next step S1407, the first angular displacement signal is output from the microcomputer 1505 as the correction target value and in next step S1408, the second integrator 1512 integrates the output of the HPF 1507 to calculate a second angular displacement signal for frequency detection. Then, in step S1409, the microcomputer 1505 finishes interruption to the overall processing under its control.
Next, with reference to the flow chart in FIG. 18, the processing of calculating a frequency from the second angular displacement signal calculated for frequency detection, deciding the vibration state of the apparatus and controlling the cutoff frequency of the variable HPF 1509 will be explained. This processing is carried out at a period different from the period of the processing in FIG. 17 (e.g., 500 [ms]).
In FIG. 18, the processing of detecting a frequency is started at step S1451 and in step S1452, the frequency calculation section 1513 calculates the vibration frequency of the apparatus based on the second angular displacement signal calculated in step S1408 in FIG. 17. The method of detecting the frequency is as described with reference to FIG. 14.
Then, in step S1453, the detected frequency is compared with a predetermined threshold (fth). When the detected frequency is higher than fth (YES), the microcomputer 1505 decides that the high frequency is detected and the process moves on to step S1454, where a cutoff frequency of the variable HPF 1509 is set based on the frequency detected in step S1452. In this case, the cutoff frequency is shifted gradually to the high frequency side. After setting the cutoff frequency of the variable HPF 1509, the process moves on to step S1456 and finishes the frequency detection and cutoff frequency setting processing.
Furthermore, when the comparison result in step S1453 is equal to or lower than the threshold (NO), the microcomputer 1505 decides that a normal vibration frequency is detected and the process moves on to step S1455, where a specified cutoff frequency during normal vibration correction is set in the variable HPF 1509 and the process moves on to step S1456 and finishes the frequency detection and cutoff frequency setting processing. The cutoff frequency set by the variable HPF 1509 is updated when the next frequency detection processing is executed and the cutoff frequency of the variable HPF 1509 is controlled as appropriate. The image sensing apparatus provided with the aforementioned vibration correction function allows image sensing with a normal handheld camera or vibration correction when mounted on a vehicle, etc.
The vibration correction function mounted on the image sensing apparatus performs similar corrections in at least two directions; vertical direction and horizontal direction. Since corrections in these two directions are similar operations, for simplicity of explanation of the conventional example, an operation in one direction was explained to represent them. Further, the driving circuit and actuator that drive the zoom lens 702 and focus lens 704, mechanism and control for exposure control are omitted in the above explanation.
According to the above described conventional example, when an image sensing apparatus such as a video camera provided with a vibration correction function is placed on a table on a ship, etc., for image sensing in an operating environment in which vibration exerted on the apparatus consists of a mixture of a low frequency vibration of reeling of the ship and a high frequency vibration due to vibration of the engine transmitted through structures of the ship, the frequency detection section 1511 detects the low frequency preferentially as explained in the flow chart in FIG. 14, failing to detect the high frequency and detect a vibration frequency of the high frequency band.
Thus, even when a mixture of low frequency and high frequency vibration which is correctable by camera shake correction is applied to the apparatus, the vibration correction frequency control section 1514 determines that the vibration is of only the low frequency, tries to adjust adaptive control for making a cutoff frequency of the variable HPF 1509 variable to the low frequency and the microcomputer 1505 outputs a correction target value for vibration correction which matches the low frequency. This prevents sufficient vibration correction of the high frequency band due to a phase delay of the high frequency band, showing a defect that the vibration suppression performance of the high frequency band is inferior to that of the low frequency band.
This reflects in such a phenomenon that when for example, an image with black and white stripe patterns is sensed and a comparison is made between a case where there is vibration of a low frequency band and a case where there is vibration of a high frequency band, the boundary between black and white appears blurred when there is vibration of a high frequency band, resulting in a defect that resolution appears deteriorated with the presence of vibration of the high frequency band.